Module 10: Large-Scale Data Stores

Advanced Architecting on AWS

S3 Advanced • Lake Formation • Athena • Redshift • Data Lake

🎯 Module Objectives

Design S3 architectures with replication, Object Lock, and Access Points
Build data lakes with Lake Formation
Query data with Athena (serverless SQL)
Understand Redshift Serverless for analytics
Implement DynamoDB Global Tables for multi-region

📦 S3 Advanced Features

Feature	Purpose	Details
Cross-Region Replication	DR, compliance, latency	Async replication. Can replicate to different account.
Same-Region Replication	Compliance copies, log aggregation	Same region, different bucket (even same account)
Object Lock	WORM compliance	Governance mode (admin bypass) or Compliance mode (immutable)
S3 Access Points	Simplified access management	Named endpoints with unique policies. Per-app access.
S3 Intelligent-Tiering	Auto-tiering	Moves objects between tiers based on access patterns

🏞️ Data Lake Architecture

S3 as the data lake foundation: Raw, processed, and curated zones
Lake Formation: Centralized governance — fine-grained access control at column/row level
AWS Glue: ETL + Data Catalog (schema discovery, crawlers)
Athena: Serverless SQL queries directly on S3 (pay per query scanned)
Redshift Spectrum: Query S3 data from Redshift without loading

Data lake zones: Landing (raw) → Processing (cleaned/transformed) → Curated (business-ready). Lake Formation acts as the security guard controlling who accesses each zone.

🔍 Amazon Athena & Redshift Serverless

Amazon Athena

Serverless SQL on S3. Presto-based. Pay $5/TB scanned. Use columnar formats (Parquet/ORC) to reduce cost. Federated queries to RDS/DynamoDB.

Redshift Serverless

Data warehouse without cluster management. Auto-scales. Pay for compute (RPU-hours). Complex analytics, joins, aggregations at PB scale.

Decision: Ad-hoc queries on S3 data? → Athena. Complex analytics with dashboards, frequent queries, PB-scale joins? → Redshift Serverless.

🌍 DynamoDB Global Tables

Multi-region, multi-active: Read and write in any region with automatic replication
Conflict resolution: Last-writer-wins (based on timestamp)
Replication latency: Typically under 1 second between regions
Use cases: Global applications, DR, low-latency reads/writes worldwide
Pricing: Replicated write capacity units (rWCU) charged per region

Pattern: Global Tables + Route 53 latency routing = users always hit the nearest DynamoDB replica with full read/write capability. True active-active at the database layer.

💻 Demo: S3 Replication & Athena

# Enable versioning (required for replication)
aws s3api put-bucket-versioning --bucket source-bucket \
  --versioning-configuration Status=Enabled

# Configure cross-region replication
aws s3api put-bucket-replication --bucket source-bucket \
  --replication-configuration file://replication.json

# Run an Athena query
aws athena start-query-execution \
  --query-string "SELECT * FROM logs WHERE date = '2025-01-15' LIMIT 100" \
  --work-group primary \
  --query-execution-context Database=analytics

🧪 Knowledge Check

Q1: A financial company must store audit logs in S3 that cannot be deleted or modified for 7 years, even by administrators. Which feature should they use?

A) S3 Versioning B) Object Lock (Governance mode) C) Object Lock (Compliance mode) D) Bucket policy deny

C) Object Lock (Compliance mode) — Compliance mode cannot be overridden by any user, including the root account. Objects are truly immutable for the retention period. Governance mode allows admin bypass.

Q2: A global e-commerce platform needs sub-second read/write latency for their product catalog in all regions. Which is the best solution?

A) RDS Multi-AZ B) Aurora Global Database C) DynamoDB Global Tables D) ElastiCache Global Datastore

C) DynamoDB Global Tables — Global Tables provide multi-region, multi-active read/write with sub-second replication. Users can write to the nearest region. Aurora Global allows reads in all regions but writes only in primary.

📝 Module 10 Summary

S3 Advanced

CRR/SRR replication. Object Lock (WORM). Access Points. Intelligent-Tiering. Lifecycle policies.

Data Lake

S3 foundation + Glue catalog + Lake Formation governance. Raw → Processed → Curated zones.

Analytics

Athena: serverless SQL on S3. Redshift Serverless: PB-scale warehouse. Use Parquet for cost savings.

Global Tables

DynamoDB multi-region active-active. Sub-second replication. Last-writer-wins conflict resolution.